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Symbols and Abbreviations 

PPI 

plan position indicator 

ARTS 

automated radar terminal system 

RWY 

runway 

ATC 

air traffic control 

SLT 

scheduled landing time 

ATIS 

automatic terminal information service 

TAATM 

terminal-area air traffic model 

ATOPS 

Advanced Transport Operating 

TIMER 

traffic intelligence for the management 

Systems 


of efficient runway scheduling 

Avg 

average 

t 

response time 

CAS 

calibrated airspeed 

t f 

transformed response time 

CDT 

command delivery time 

VAR 

variation of the Earth’s magnetic field 

CRT 

cathode ray tube 

x,y,z 

Cartesian coordinate system 

DEN 

Denver’s Stapleton International 

a 

percent of significance of F test 


Airport (or Denver Vortac) 

P 

shape parameter of Weibull distribution 

DICE 

direct-course error 

1 

location or threshold parameter of 

ETA 

estimated time of arrival 

Weibull distribution 

FAA 

Federal Aviation Administration 

V 

scale parameter of Weibull distribution 

FMS 

flight management system 

V 

mean value 

H 

rate of descent, ft/sec 

a 

standard deviation 

IAS 

indicated airspeed 

<?c 

standard deviation of controller’s 

IFR 

instrument flight rules 

message-delivery-time error relative 
to TIMER expected delivery time 


ILS 

instrument landing system 


IMC 

instrument meteorological conditions 


runway delivery- time-error standard 
deviation for a single aircraft 

L 

left 

Oi 

runway interarrival-time-error stan- 

MOTAS 

Mission-Oriented Terminal- Area 
Simulation 

<jp 

dard deviation for an aircraft pair 
pilot-in-the-loop/TIMER algorithm, 

MSL 

mean sea level 

time-error standard deviation for a 

n 

number of cases 


single aircraft 

O.M., OM 

outer marker 

4D 

four-dimensional (x, z , and time) 
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Summary 

A real-time simulation study was conducted of a 
time-based, extended-terminal-area air traffic control 
(ATC) concept called traffic intelligence for the man- 
agement of efficient runway scheduling (TIMER). 
The principal objective of the study was to measure, 
under realistic full-workload conditions, the perfor- 
mance and reaction of flight crews flying TIMER- 
assisted approaches in a conventional electromechan- 
ical cockpit without cathode ray tube (CRT) displays 
or a four-dimensional (4D) flight management sys- 
tem. Additional experimental objectives were to ver- 
ify earlier fast-time TIMER study results and to ob- 
tain data for the validation or refinement of computer 
models of pilot/airborne performance. A real-time 
ATC simulation with the TIMER algorithm embed- 
ded was linked to a DC-9 cockpit simulator. These 
facilities together with a certified ATC controller, 
pseudo pilot, and subject professional airline crews 
formed the basis for a total system simulation for 
realistic crew-in-the-loop experiments. 

Previous TIMER fast-time results in NASA 
TP-2870 indicated a runway-threshold, interarrival- 
time-error standard deviation of about 12 sec for 
pairs of non-4D aircraft with computer aiding and 
knowledge of the aircraft final- approach speed. As 
a point of reference, a report by Martin and Wil- 
lett in 1968 measured 26.5 sec for the interarrival- 
time-error standard deviation of manual control with 
no computer aiding. Based on the measured real- 
time, piloted-simulation delivery error at the runway 
threshold, a system interarrival-time-error standard 
deviation was determined to be in the range of 10.4 
to 14.1 sec. This real-time outcome supports the 
12 sec predicted by earlier TIMER fast-time simu- 
lations and demonstrated the evolutionary features 
of the concept. (That is, the concept bridges the 
gap between conventional aircraft cockpits having a 
manual voice-linked ATC system and advanced air- 
craft cockpits in a data-link environment with fur- 
ther ground automation.) Other real-time system 
performance parameters measured include approach 
speeds, response time to controller’s turn instruc- 
tions, bank angles, and ATC controller s message- 
delivery-time errors. These data will enable ATC 
researchers to verify and refine computer models of 
the pilot/airborne system and the controller’s perfor- 
mance. 

1.0 Introduction 

A description and a detailed fast-time evalua- 
tion of a time-based, extended-terminal-area air traf- 
fic control (ATC) concept called traffic intelligence 
for the management of efficient runway scheduling 


(TIMER) was presented in reference 1. The results 
identified and showed the effects and interactions of 
key system variables. The TIMER concept was de- 
signed for evolutionary integration into the manual, 
voice-linked ATC system and for handling both four- 
dimensional (4D) flight management system (FMS) 
equipped aircraft and non-4D-equipped aircraft. As 
was stated in reference 1, it is anticipated that when a 
terminal-area, time-based ATC system is first intro- 
duced, many, if not most, of the aircraft will not be 
equipped with a terminal-compatible 4D FMS. Un- 
til most of the aircraft are equipped with a flexible, 
terminal-compatible 4D FMS, the performance of a 
time-based system will be constrained by non-4D air- 
craft performance. Thus, the runway delivery-time 
performance precision achievable with conventional 
aircraft not equipped with 4D FMS is a key issue 
relating to the system capacity of an initial terminal- 
area, time-based flow control system such as TIMER. 

Fast-time simulation results (ref. 1) indicated 
that with computer aiding, which has aircraft final- 
approach-speed information, a runway interarrival- 
error standard deviation of about 12 sec could be 
achieved for non-4D traffic. As a point of refer- 
ence, a field study (ref. 2) conducted by the Federal 
Aviation Administration (FAA) in New York mea- 
sured 26.5 sec for the interarrival-error standard de- 
viation of manual control with no computer aiding. 
One of the objectives of this study was to validate the 
fast-time results by assessing the performance and 
reaction of flight crews under realistic full-workload 
conditions when flying a TIMER-assisted approach 
in a conventional electromechanical cockpit without 
a 4D FMS. Another objective was to obtain data on 
pilot /aircraft performance so that computer models, 
used in TIMER and other ATC simulations, could 
be validated or refined. The primary area of inter- 
est is the final-approach performance in a TIMER 
environment. 

Several facilities were integrated and used to con- 
duct the simulation study. The real-time version 
of the terminal area air traffic model (TAATM) de- 
scribed in reference 3 with the TIMER algorithm em- 
bedded was linked to a DC- 9 cockpit simulator via 
the Langley Mission- Oriented Terminal- Area Simula- 
tion (MOTAS) Facility. Additional facilities included 
controller work stations, a voice communication link, 
and a landing -scene generator. These facilities to- 
gether with a certified ATC controller, pseudo pilot, 
and professional airline crews as subjects formed the 
basis of a total system simulation for real-time crew- 
in-the-loop experiments. 

Several simulated instrument flight rules (IFR) 
approaches, based on a runway 26L configuration at 
Denver’s Stapleton International Airport, were flown 



from two different routes by each of the airline crews. 
A series of data runs were conducted with each crew 
requested to react to controller instructions as they 
normally would on an IFR approach. Each crew was 
then briefed on the TIMER concept and how it was 
being applied; then another series of simulated ap- 
proaches were performed and the differences were 
compared. TIMER operations and controller activi- 
ties were the same for both series of data runs. The 
runway-threshold error of each approach was mea- 
sured as well as the pilot /aircraft system response to 
speed and turn commands. Individual crew reactions 
were obtained via questionnaires and comments. 

2.0 TIMER Concept Overview 

2.1 TIMER Concept Features 

TIMER is an extended-terminal-area flow control 
concept that begins its control in en route airspace 
at the horizon of control. The major operational 
features of the TIMER concept are shown in figure 1. 
The principal features are summarized as follows: 

1. The arrival stream into the extended terminal 
area is derandomized at the horizon of control by 
establishing a proposed aircraft landing sequence 
and building a list of aircraft scheduled landing 
times (SLT’s) based on separation criteria (events 
® and © of fig. 1). The desired metering- fix 
time as a result of the assigned landing time is 
also determined. 

2. Nominal estimated times of arrival used in step 1 
are based on representative aircraft performance 
models. From using these models and predicted 
winds, a ground-computed trajectory is deter- 
mined to meet the aircraft assigned scheduled 
landing time (events © and © of fig. 1). 

3. Computer-generated assistance is given to the 
controller to help him meet aircraft target times 
based on the trajectory calculations. The param- 
eters determined are the en route cruise speed, 
the time to initiate as well as the Mach/CAS 
speeds to fly a flight-idle-thrust descent, and the 
terminal segment speeds and headings. 

4. Adjustments to the scheduled landing times and, 
if necessary, changes in the landing sequence are 
made to accommodate errors and anomalies in 
factors such as wind, navigation, airspeed, and 
heading which affect the SLT of either the own 
aircraft or the preceding aircraft. These sched- 
ule adjustments or controller-action points occur 
at the following points shown in figure 1: the 
metering fix, the speed adjustment points, and 
the fine-tuning region. The landing sequence 
is fixed before aircraft arrive in the fine-tuning 
region. 


5. The aircraft trajectory is fine tuned in the final- 
approach region in order to meet the aircraft 
final scheduled landing time with limited time 
error. The performance in the final- approach fine- 
tuning region is the primary focus of this paper. 

A more complete description of the TIMER 
concept is furnished in reference 1. References 4 
through 9 describe some of the recent, closely 
related research and development activity in the 
area of extended-terminal-area, time-based air traffic 
control. 

2.2 Fine-Tuning Region 

The TIMER fine-tuning region is defined by the 
boundaries of the vector heading from the aim point 
for the eastern arrivals (event © in fig. 1) and by 
the boundaries of the downwind-to-base turn for the 
western arrivals (event © in fig. 1). The dashed lines 
in both figures 1 and 2 indicate the boundaries of the 
fine-tuning region. Within this region the computer- 
aided fine-tuning maneuvers consist of timing both 
the turn-to-base maneuver (event © in fig. 1) and the 
turn-to- final maneuver (event ©). In keeping with 
the evolutionary nature of the TIMER concept, the 
design was configured to be similar in geometry and 
procedures to the conventional approach performed 
today. That is, the pilot would not be able to distin- 
guish between a TIMER-assisted final approach and 
a conventional radar, manual-controlled approach. 

The fine-tuning process is based on a regularly 
updated, estimated time of arrival (ETA) calculation 
that displays how early, relative to its SLT, the 
aircraft would be if its turn instructions were issued 
immediately. The difference between the SLT and 
the ETA is referred to as the direct-course error 
(DICE) value. DICE gives more information than 
a straight clock countdown display that indicates 
only the remaining time to issue the turn command. 
With expected communication and response times 
of both the controller and pilots factored in, the 
data tag of each aircraft on the controller display is 
enhanced to indicate when, and to what heading, the 
controller would vector the aircraft for the base and 
localizer intercept segments. The fine-tuning region 
must accommodate minor schedule changes due to 
other aircraft errors, wind estimate errors, or own 
aircraft flight errors that have accumulated since the 
last speed control point. 

3.0 Experimental System 

3.1 Experimental Facilities 

A complex system simulation using the Denver 
Stapleton approach routes was assembled to provide 
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a realistic and dynamic environment for measuring 
crew-in-the-loop, aircraft approach performance in 
a TIMER environment. A real-time version of the 
terminal-area air traffic model (TAATM) with the 
TIMER algorithms embedded interacted with a cer- 
tified ATC controller by means of a controller sta- 
tion and plan position indicator (PPI) to provide the 
ATC environment and scenarios. The Langley DC-9 
Full- Workload Simulator cockpit provided the vehi- 
cle for airline crew interaction. The computer data 
interfaces, voice links, and controller workstations of 
the Langley Mission-Oriented Terminal- Area Simu- 
lation (MOTAS) Facility (ref. 10) linked the TIMER 
algorithms and cockpit simulator interactively. 

3.1.1 Mission- Oriented Terminal-Area 

Simulation 

The Langley Mission- Oriented Terminal- Area 
Simulation (MOTAS) Facility shown in figure 3 (and 
discussed in ref. 10) is a sophisticated system sim- 
ulation capability that provides an environment in 
which research studies can be conducted in flight 
management and flight operations with a high de- 
gree of realism. This facility provides a flexible and 
comprehensive simulation of the airborne, ground- 
based, and communications aspects of the airport 
terminal-area environment. The major elements of 
the MOTAS facility are an airport terminal-area en- 
vironment model, several aircraft models and sim- 
ulator cockpits, four pseudo-pilot stations, two air 
traffic controller stations, and a realistic air /ground 
communications network. In addition, the MOTAS 
facility can be linked to the NASA Wallops Flight Fa- 
cility at Wallops Island, Virginia, by telephone data 
communication lines or satellite to allow interaction 
with live aircraft for actual flight tests. The airport 
terminal-area environment model gives a current rep- 
resentation of Denver’s Stapleton International Air- 
port and surrounding area using either the TIMER 
automation aids to the controller or present-day man- 
ual control. In addition, the model simulates radar 
systems, navigation aids, wind conditions, etc. 

The MOTAS facility combines the use of several 
aircraft cockpit simulators and pseudo-pilot stations 
for flying aircraft in the simulated airport terminal 
area. The facility is presently operational with the 
Transport Systems Research Vehicle (TSRV) Simu- 
lator, the General Aviation Simulator, and the DC-9 
Full- Workload Simulator. These aircraft cockpit sim- 
ulators allow entire crews to fly realistic missions in 
the airport terminal area. Simulated aircraft can also 
be controlled through the use of the pseudo-pilot sta- 
tions. The operators of these stations can handle 
up to 15 aircraft simultaneously by inputting control 
commands to change airspeed, altitude, direction, 


etc. The final major component of the facility is the 
air traffic controller stations, which are described in 
section 3.1.3. 

3.1.2 The DC-9 Full-Workload Simulator 

The transport aircraft cockpit (fig. 4) used in this 
study was a fixed-base full-workload simulator out- 
fitted as a DC-9 aircraft. A television model board 
(fig. 5) provided the out-of-window landing scene for 
the crew, which included visibility effects and ap- 
proach lighting systems used to simulate Category I 
landing conditions for this study (ref. 11). The air- 
craft dynamics modeled were those of the McDonnell 
Douglas DC-9 Series 30 aircraft. A full comple- 
ment of cockpit electromechanical instruments was 
provided, with all major systems functional includ- 
ing autopilot, dual flight directors, and navigation 
and communication radios. Subsystems, such as hy- 
draulics and electrical systems, were modeled to the 
extent necessary to provide normal in-flight opera- 
tions and readouts to the crew. 

The aircraft cockpit and terminal- area simula- 
tion programs were run simultaneously on separate 
Control Data CYBER 175 computer systems that 
communicated with each other through extended 
core storage. This allowed various aircraft data, such 
as x,y,z positions, airspeed, bank angle, etc., from the 
DC-9 program to be sent to the terminal-area pro- 
gram for processing. The multifrequency radio com- 
munications were simulated with selectable voice- link 
channels between the cockpit radio tuning heads and 
the ATC controller’s radar display consoles of the 
terminal-area simulation. 

3.1.3 Simulation Air Traffic Controller Stations 

The air traffic controller’s interaction to the cock- 
pit was provided using the facilities of the Mission- 
Oriented Terminal- Area Simulation (MOTAS). The 
MOTAS components used for the simulation are de- 
scribed below. 

As shown in figure 6, two controller stations (ATC 
stations 1 to the left and 2 to the right) were equipped 
with a plan position indicator (PPI) on which aircraft 
position information is displayed. Each station had 
a communications system for the verbal exchange of 
information between the controller, the subject air- 
craft (DC-9 cockpit), and the pseudo pilot as indi- 
cated in figure 7. Additionally, ATC station 2 is 
equipped with an electronic data tablet for inter- 
action with TIMER-generated commands that ap- 
pear on the PPI. 

The controller displays are simulated by an Evans 
& Sutherland Multi Picture System with a CRT mea- 
suring 23 in. in diameter and mounted vertically 
(fig. 6). The display is configured with a video map of 
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the Denver terminal area (fig. 8). Status information 
for the instrument approach in use, the altimeter set- 
ting, and the automatic terminal information service 
(ATIS) code were presented in the lower portion of 
the display. 

An aircraft position is indicated by an elongated 
delta symbol with two trailing chevrons that provide 
past-position information of the aircraft track. An 
associated alphanumeric data block is connected to 
the aircraft symbol with a leader line; the specifics of 
the alphanumeric data block are addressed in a later 
section. 

Several components comprise the communica- 
tions system. Both ATC stations are equipped with 
an audio control panel and four preset frequencies. 
The audio panels are the interface between the con- 
troller headsets and a switching system that searches 
for a match of the frequency selected at the controller 
stations and at the cockpit. A voice disguiser linked 
to the audio panel serving ATC station 1 (the left 
station in fig. 7) provided the capability for varying 
the voice output of that station, thus creating for 
the DC-9 cockpit the environment of different pilots 
responding to ATC commands. 

3.2 Experimental Conditions and 

Interactions 

3.2.1 Terminal- Area Conditions 

The approach routes of Denver’s Stapleton 
International Airport operating in a runway 26L 
landing configuration (fig. 2) were used assuming in- 
strument meteorological conditions (IMC) weather 
procedures. In anticipation of real-time experiments 
with the ATOPS Boeing 737 research aircraft oper- 
ating at the Wallops Flight Facility, the approach 
paths were modified in the TIMER to reflect the 
corresponding magnetic variation and altitudes at 
Wallops. Thus, the terminal-area altitudes were re- 
duced by approximately 5000 ft from the Denver alti- 
tudes. The outer-marker intercept altitude and run- 
way elevation were set to 1900 ft and 40 ft MSL, 
respectively, and the active runway was designated 
as 28L. Hereinafter, the simulated airport will be re- 
ferred to as Denver/ Wallops. 

The area of interest in this study was the final- 
approach performance of the conventional electro- 
mechanical, non-4D-equipped aircraft in the TIMER 
environment. Since the southern approach rout- 
ings (BYSON and KIOWA) are geometrically simi- 
lar to the northern routings (DRAKO and KEANN), 
only the latter two were used for experimental data 
flights in the cockpit simulator. However, computer- 
simulated traffic controlled by TIMER was flown 
from all four approach routes. 


A linear wind model using statistical coefficients 
for an average Denver area wind was used in all runs. 
The simulated wind velocity at ground level was 
7.9 knots from 277° with a speed gradient of 2.368 
knots per 1000 ft. In this study the wind direction 
was constant at all altitudes. 

3.2.2 Approach Paths and Procedures 

The expected flight paths for the two inbound 
routes used by the cockpit simulator in the study are 
depicted in the simplified approach charts shown in 
figures 9 and 10. The charts show applicable radio 
frequencies, nominal indicated airspeeds, magnetic 
headings, and MSL altitudes in the final-approach 
area. For each experimental flight either the DRAKO 
or KEANN approach was chosen from the subset of 
simulated DC-9 flights generated in a fast-time run 
under simulation conditions identical to those in the 
real-time experiment. Thus, the initial conditions for 
each approach within the subset were slightly differ- 
ent because of variations in assigned airspeeds and 
the error models in the TIMER program. Each crew 
used the same subset of initial conditions. The ac- 
tual initialization points were selected to be 1 minute 
prior to either the turn to downwind on the DRAKO 
approaches or to the turn to base in the vicinity of 
the FLOTS intersection on the KEANN approaches. 

The events that occurred during the approaches 
can be described by referring to the labeled areas 
in figure ll. For initialization (area I), the posi- 
tion, speed, heading, and descent rate of the sim- 
ulated target aircraft were transmitted to the DC-9 
simulation program via computer memory shared by 
the TIMER and DC-9 programs while the TIMER 
was put in a hold mode. After the aircraft simulator 
was aerodynamically trimmed and the two programs 
were returned to real-time operation mode, the pilot 
was expected to maintain the initial airspeed and to 
track inbound to the DEN VORTAC. There was an 
initial communication exchange between the cockpit 
and the feeder controller to establish communications 
and to verify descent clearance to 6000 ft MSL. 

At the areas labeled ©on the DRAKO approach 
and ©on the KEANN approach in figure 11, the 
TIMER generated a vector message and time at 
which the controller should initiate the corresponding 
turn command. This turn was to the downwind head- 
ing for the DRAKO approach and to a variable base- 
leg heading for the KEANN approach. The TIMER 
algorithm computed the heading compensation for 
wind to maintain the desired ground tracks. All re- 
quested headings were rounded to the nearest mul- 
tiple of 10°. The next events were the handoff from 
the feeder controller to the final controller on the 
DRAKO approach (H) or a speed change to 170 knots 
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(S) immediately followed by the handoff (H) on the 
KEANN approach. 

After communication was established with the 
cockpit, the final controller issued altitude clear- 
ances at his discretion to descend the subject aircraft 
to 1900 ft prior to intercepting the final-approach 
course. On the DRAKO approach, a speed adjust- 
ment (170 knots) was generated as the aircraft ap- 
proached a specified location (S) on the downwind 

leg- 

When approaching or when within the fine-tuning 
region, bounded by the dashed lines in figure 11, 
the direct-course error (DICE) with respect to the 
runway was periodically computed and displayed to 
the controller. As described in section 2.1, two DICE 
turns were generated for the DRAKO approach (B 
and F), and one DICE turn was generated for the 
KEANN approach (F). In conjunction with the last 
DICE turn, clearance was issued for the ILS runway 
28L approach. As part of the instrument approach 
clearance, the pilot was instructed to contact the 
tower at the outer marker. For the remainder of the 
flight, the pilot continued on a normal ILS approach 
to the runway. Each run was flown to runway 
touchdown and terminated after starting the rollout. 

3.2.3 Controller/Pseudo-Pilot Function 

Arrival traffic inbound to the simulated Denver/ 
Wallops Airport transits two terminal control sectors 
(the feeder sector and the final sector). The subject 
aircraft (the DC-9 simulator) was initialized in the 
feeder sector between 10 and 15 n.m. from the final 
control sector. Control instructions issued in this sec- 
tor consisted of TIMER-generated commands that 
included turns off the initial inbound routing and 
speed reductions. 

At the beginning of the run, the individual at 
ATC station 1 assumed the role of controller to 
the DC-9 simulator and other sector traffic; the 
individual at ATC station 2, a certified air traffic 
controller, acted as a pseudo pilot. As the aircraft 
entered the final sector, 3 to 4 minutes into the run, 
these roles were reversed for the duration of the run. 
Transfer of control to the final controller occurred 
after the subject aircraft received the initial turn off 
the inbound route (i.e., an easterly heading when 
on the DRAKO approach or a heading that varied 
between 150° and 200° for the KEANN approach). 
The transfer of communications occurred when the 
aircraft was instructed to “Contact Approach on 
125.3,” at which time the aircraft contacted the final 
sector. 

In the final control sector, the aircraft was vec- 
tored to the final-approach course at the proper alti- 
tude using TIMER-generated fine-tuning commands 


to reduce arrival-time error over the runway thresh- 
old. Aircraft were instructed to contact the tower 
at the outer marker; the controller then assumed the 
role of the tower controller and provided communi- 
cations normally encountered by aircraft monitoring 
the tower frequency. To enhance realism, the land- 
ing clearance for the subject aircraft was issued at 
varying points along the final- approach course. 

The procedures used for the ATC simulation were 
taken from the air traffic control manual (ref. 12) 
and were identical to those used in actual practice. 
The phraseology used and the clearances issued con- 
formed both in content and format to those currently 
used in the real-world ATC system. 

3.2.4 Controller Interface to TIMER 

The TIMER-generated commands were presented 
as part of the alphanumeric data block associated 
with each aircraft. These data blocks were com- 
posed of 10 fields of text as described in figure 12. 
Fields 2 through 6 (shown shaded in fig. 12) con- 
tained information common to that found in termi- 
nal ARTS III data blocks. All other fields contain 
information that was unique to the TIMER program 
and not found in current ATC terminal data blocks. 
Field 1 was used to display the DICE value of air- 
craft in the fine-tuning region. The DICE value (de- 
scribed in section 2.2) decreased at a rate that was 
dependent on the ground track and speed of the air- 
craft. Field 7 displayed the TIMER-recommended 
heading and field 8 displayed the recommended air- 
speed. Fields 9 and 10 provided time-reference infor- 
mation with respect to the time at which the com- 
mand should be delivered, the command delivery 
time (CDT). 

For aircraft flying outside the fine-tuning region, 
the CDT’s for speed and vector commands were 
computed using estimated ground speeds and tracks 
with respect to specified geographic locations. Thus, 
the required CDT’s were generated well in advance of 
the aircraft reaching these locations. Within the fine- 
tuning region, however, the flight path geometry was 
somewhat variable. Therefore, the desired vectors 
for turning to the base and final-approach legs were 
not shown in the aircraft data tag until the actual 
clearance was generated, which was approximately 0 
to 4 sec prior to the associated CDT. 

The time remaining to the CDT provided useful 
information to the controller in integrating the is- 
suance of the command into his workload. Should 
the command not be issued on time, a “countup’ 
feature provided information needed to compensate 
for delinquency in delivery of the command. For ex- 
ample, if the program generated a left turn to 350° 
and the controller was 5 sec late in transmitting the 
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command, a turn to 340° might be issued in- 
stead if the controller judged that necessary for 
compensation. 

The controller disabled the command by use of 
an electronic data tablet and an associated pen. The 
pen was moved upon the data tablet, which corre- 
spondingly moved a slaved “X” on the traffic dis- 
play. When the “X” was placed over the appropriate 
data block and the pen switch depressed, the com- 
mand and the time reference information were dis- 
abled. The data block then returned to its original 
state. The time at which the command and the time 
information were disabled was recorded and used to 
determine if the early or late delivery of a command 
might have influenced the arrival time of an aircraft 
at the runway threshold. 

Figure 13 depicts the evolution of a data block 
that contains a vector control command. Figure 14 
shows an example of a DICE countdown and ensuing 
vector command. 

3.2.5 Subject Crew Profile and Experimental 

Task 

Eight professional DC-9 rated pilots, four from 
each of two major U.S. airlines, served as test sub- 
jects for this study. Each pilot served as the captain 
and then the first officer for a crew, thus allowing 
eight full crews to be formed from the eight pilots. A 
crew was defined in the test matrix by the pilot serv- 
ing as captain during the flights. The captain han- 
dled all flying duties during the flight, and the first 
officer handled communication and navigation radio 
functions, checklists, flaps, gear, etc. The individ- 
ual subjects had experience in a variety of jet trans- 
port aircraft including the McDonnell Douglas DC-9 
and DC-8; the Boeing 727, 737, and 757; the British 
Aerospace BAG 111; and the Lockheed L-1011 and 
C-141. The total jet transport experience ranged 
from a low of 4800 hours to a high of 13 300 hours, 
with an average of 6800 hours. The DC-9 experi- 
ence ranged from a low of 1500 hours to a high of 
6000 hours, with an average of 3500 hours. 

The task presented to each crew was to fly either 
the KEANN route or the DRAKO route of the simu- 
lated Denver/Wallops Airport with a final approach 
and landing to runway 28L. The weather conditions 
simulated were those of Category I instrument flight 
rules with a ceiling of approximately 200 ft and a run- 
way visual range of approximately 0.5 n.mi. Thus, 
the out-of-window visual scene presented an image 
of being in the clouds until the breakout occurred, at 
which point the crew would be able to see the run- 
way and approach lights. The crew was requested 
to fly the aircraft as they normally would when us- 
ing manual controls, manual throttles, and a flight 


director. Figure 15 presents a simplified cockpit sim- 
ulator procedure that occurred during the setup for 
each run and during each flight. The crew was to per- 
form all normal flight-deck tasks associated with fly- 
ing through a terminal area and making an approach 
and landing including normal checklists. (A normal 
checklist for each crew was that used by their par- 
ticular airline.) The crew was requested to respond 
in a normal manner to air traffic control instructions 
for speed and altitude changes and vectoring maneu- 
vers. As part of normal operations, the crew was 
required to accomplish all radio-communication fre- 
quency changes when instructed by air traffic control 
as the aircraft was handed off from sector to sector. 
Figures 16 and 17 present the navigation chart and 
ILS approach plate, respectively, used by the crews 
for this study. 

3.2.6 Experimental Matrix and Parameters 

The experimental organization or design is per- 
haps best shown in the following table: 


Experimental 

condition 

Simulation 

flights 

Crew number 

1 . 

. . 71 . . 

.8 


Practice 

X. 

..X. 

.X 

Before 

Run 1 

X. 

.X. 

.X 

TIMER 

Run 2 

X. 

.X. 

.X 

Briefing 

Run 3 

X. 

.X. 

.X 


Run 4 

X. 

.X. 

X 


Practice 

X. 

.X. 

.X 

After 

Run 1 

X. 

.X. 

X 

TIMER 

Run 2 

X.. 

.X. 

.X 

Briefing 

Run 3 

X.. 

.X. 

.X 


Run 4 

X.. 

.X. 

.X 


Run 5 


X 



A total of eight crews served as subjects in the 
TIMER/DC-9 experiment. A set of runs was made 
wherein each crew was requested to react to con- 
troller instructions as they normally would in instru- 
ment meteorological conditions (IMC) when transi- 
tioning to and executing an instrument approach. 
This series was called the “before-TIMER-briefing” 
runs. Each crew was given a full practice approach 
and then performed 4 data runs for a total of 32 
before-briefing data runs. 

Each crew was then briefed on the TIMER con- 
cept and how it was being applied, stressing that 
the performance depended on timely and consis- 
tent execution of ATC request. The presumption is 
that the after-briefing case represented pilots with a 
knowledge and awareness of the TIMER system, i.e., 
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pilots who were motivated and attentive to ATC in- 
structions. Another series of simulated approaches 
was flown. This second set of runs was labeled the 
“after-TIMER-briefing” runs. TIMER operation and 
controller procedures were consistent for both before- 
and after- briefing runs. Some crews performed 4 data 
runs and others performed 5 for a total of 37 after- 
briefing data runs. The following data were taken 
during the experiment: 

Runway-threshold time error 
Final-fix time error 

DC-9 x,y,z position as a function of time 
Pilot’s response time to ATC turn instructions 
DC-9 bank angle during turns 
Captain’s questionnaire rating 
First officer’s questionnaire rating 
Controller’s final-turn message-delivery-time 
error 

In a human-in-the-loop experiment there is 
always the concern that the learning-curve 
phenomenon will have an effect that is falsely 
attributed to an experimental parameter. The fol- 
lowing steps were taken to address this concern: 
(1) The DC-9 Full- Workload Simulator cockpit was 
utilized, (2) DC-9 certified, professional airline pi- 
lots were used as test subjects, (3) standard cockpit 
procedures were used during the flight, (4) no un- 
usual or nonstandard items were in the cockpit such 
as experimental displays, and (5) the DC-9 simula- 
tor was developed with the technical guidance of a 
senior pilot from one of the two airlines providing 
pilots for this study. Also, before data were taken, 
each crew was briefed and given familiarization time 
in the DC-9 cockpit before performing a full-blown 
practice run. In addition, the performance data and 
crew questionnaire were cross-checked to determine 
if there was a progressive performance improvement 
due to a learning effect. Data from the crews’ first 
data runs were compared with the fourth data runs 
within the before-briefing data set. 

4.0 Real-Time Simulation Results and 
Discussion 

Measurements were taken of the delivery-time 
precision achieved at the runway threshold and 
final-approach fix by a flight crew under realisti- 
cally simulated full-workload conditions when fly- 
ing a TIMER- assisted approach in a conventional 
electromechanical cockpit without a 4D FMS. Other 
real-time system performance parameters measured 
include final-approach speeds flown, approach routes 
and speeds flown, response time to controller’s turn 
instructions, bank angles employed, and ATC con- 
troller’s message-delivery- time errors. In addition, 


aircraft-crew questionnaire data were collected after 
each simulated approach. 

4.1 Piloted Simulation Performance 

4-1.1 Delivery- Time Precision at Runway 

Threshold 

The runway delivery-time precision achievable 
with conventional aircraft not equipped with a 4D 
FMS is a primary parameter of interest in this 
study. The cockpit/pilot airborne system perfor- 
mance was measured under crew-in-the-loop, real- 
istic full-workload conditions. Figures 18 and 19 
show the results before and after pilot briefing, re- 
spectively, of the cockpit simulator delivery-time er- 
rors at the runway threshold. The time errors are 
defined as the difference in threshold crossing times 
between the SLT and the recorded DC-9 simulator. 
It should be noted that the TIMER SLT’s are dy- 
namic, and thus the errors are relative to the last SLT 
used that occurs prior to the turn-to-final maneu- 
ver. Under the experimental conditions measured, 
the ATC controller’s time error in delivering the 
final turn message, relative to the delivery time in- 
dicated by TIMER, was removed to isolate the com- 
bined TIMER system and pilot /cockpit simulator de- 
livery precision. 

In a crew-oriented experiment, there is always the 
concern that an effect attributed to an experimental 
parameter might, in fact, be caused by a learning- 
curve phenomenon. As mentioned earlier, several 
steps were taken to eliminate this effect. As a cross- 
check, the standard deviation of the DC-9 runway 
time errors for the first data run of each crew before 
briefing was computed and compared with the time 
errors of that crew’s fourth data run before briefing. 
The standard deviation was 7.9 sec for the crews’ 
first data run and 8.6 sec for the crews’ fourth data 
run. Thus, there appears to be no progressive im- 
provement in runway time-error standard deviation 
within the before-briefing data set that would indi- 
cate a learning effect. 

The standard deviation of the DC-9 runway 
delivery-time error was computed for the runs both 
before and after pilot briefing. There are two signifi- 
cant and closely related issues here. One is whether 
there is a difference in the delivery precision before 
and after briefing. The other issue is the range of 
the delivery precision that can be expected with a 
TIMER-assisted approach to the runway. 

With no time-error contribution by the ATC con- 
troller and with the assumption that the before- 
briefing case represents today’s typical airline crew 
response, the data indicated a non-4D, single- aircraft, 
runway time-error standard deviation of 9.7 sec. The 
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experimental assumption was that the after-briefing 
case represented pilots with a knowledge and aware- 
ness of the system goals (i.e., pilots with attention 
and motivation to promptly and consistently respond 
to ATC instructions). The single-aircraft, runway 
time-error standard deviation computed for after- 
briefing runs was 7.0 sec. 

The statistical F test was used to evaluate 
whether there was a difference in the standard de- 
viation in before- and after-briefing runway time er- 
ror. The null hypothesis for the F test contends 
that the ratio of the variance is 1 (i.e., ^before — 
cr a f ter ) at some significance a. For the before- and 
after-briefing standard deviations computed, the null 
hypothesis can be rejected at a significance of 5.6 per- 
cent. A confidence of 94.4 percent is slightly less con- 
clusive than the desirable 95 percent; however, the 
statistical result does support the proposition that 
there was a small difference in the single-aircraft, 
runway time-error standard deviation before and af- 
ter briefing. Though there appears to have been a 
slight reduction in the delivery precision after brief- 
ing, what is significant is how well the system works 
even with normal pilot response to ATC instructions. 
In fact, 7.0 to 9.7 sec can be treated as the range 
of single-aircraft standard deviations to be expected 
from airline pilots manually controlling an aircraft in 
response to verbal instructions from a final-approach 
controller who has computer aiding. 

If Gaussian distributions are assumed, the total 
system (TIMER algorithms, pilot-in-the-loop, and 
controller) runway time-error variance for a single 
aircraft is 

(T d =a p + (T c (!) 

where 

Op pilot-in-the-loop/TIMER algorithm, 

time-error standard deviation for a single 
aircraft 

o c standard deviation of controller’s 

message-deli very- time error relative to 
TIMER expected delivery time 
Assuming Gaussian distributions and a standard de- 
viation of 2.3 sec in the ATC controller’s message- 
delivery-time error (from section 4.2), the total sys- 
tem delivery-time-error standard deviation of single 
aircraft at the runway threshold would be 10.0 and 
7.4 sec before and after briefing, respectively. 

As described in reference 1, if Gaussian distri- 
butions are assumed, the relation between single- 
aircraft, runway time-error standard deviation and 
aircraft-pair, runway interarrival-time-error standard 
deviations is given by 

a d ( 2 ) 


where 

o d runway deli very- time-error standard 
deviation for a single aircraft 

runway interarrival-time-error standard 
deviation for an aircraft pair 

Using equation (2), the runway time-error stan- 
dard deviations of 7.4 and 10.0 sec for single aircraft 
translate to interarrival-time-error standard devia- 
tions of 10.4 and 14.1 sec for corresponding aircraft 
pairs. These interarrival values bracket the 12 sec ob- 
tained in the earlier fast-time simulation and there- 
fore support the reference 1 findings. As a point 
of reference, an FAA field study (ref. 2) measured 
a runway interarrival-time-error standard deviation 
of 26.5 sec for the aircraft pairs using manual con- 
trol with no computer aiding. Thus, both the pre- 
vious fast-time results and the real-time human- in- 
the-loop simulation results of this study show the 
potential of a time-based, terminal flow control sys- 
tem with controller aids such as those obtained in 
TIMER. With ground system knowledge of the air- 
craft final- approach speed, the runway interarrival- 
time-error standard deviation of the non-4D aircraft 
pair could be reduced to a region between 10.4 and 
14.1 sec. 

4.1.2 Delivery- Time Performance at 

Final- Approach Fix 

Since the DC-9 simulator was configured with a 
known and constant landing weight for all the ap- 
proaches, the pilot’s flip-chart final- approach speed 
was also constant. The modeled and reported surface 
wind was also constant. Given these conditions, the 
time-error standard deviations at the final-approach 
fix (outer marker) should be in the proximity of those 
measured at the runway threshold. Figures 20 and 21 
show the results before briefing the pilot and after 
briefing the pilot, respectively, of the time errors for 
the DC-9 simulator at the final-approach fix. The 
be fore- briefing standard deviations were 9.1 sec at 
the final-approach fix as compared with 9.7 sec at 
the runway threshold. The after-briefing standard 
deviations were 5.4 sec at the final-approach fix as 
compared with 7.6 sec at the runway threshold. The 
slightly higher standard deviations at the runway 
threshold relative to those at the final-approach fix, 
for both the before- and after-briefing cases, result 
from the variation in final-approach speeds flown by 
the individual pilots. The next section will address 
the final-approach-speed performance. 

4.1.3 Final- Approach- Speed Performance 

The final-approach speed is defined as the sta- 
bilized speed that the pilot reduces to on the glide 
slope when flying an instrument approach between 
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the final-approach fix and the runway. For each air- 
craft type there exists a final approach and landing 
speed (typically 1.3 times the stall speed) which, for 
the recommended flap setting, is a function of air- 
craft landing weight. The weight/speed information 
is typically contained in a tabular form in the pilot’s 
takeoff- and landing-speeds flip chart. Generally, the 
recommended rule in airline training manuals is to 
add one- half the surface headwind plus the gust value 
to the indicated flip-chart speed. This resultant air- 
speed value will be referred to as the “expected final- 
approach speed.” However, there is some variability 
in the wind adjustment from pilot to pilot. 

Reference 1 showed that large variations from the 
expected final-approach speed would have a signif- 
icant impact on runway interarrival error. There- 
fore, determining the extent of actual pilot vari- 
ation in final-approach speed from that expected 
is an important parameter in assessing a terminal- 
area, time-based flow control system such as TIMER. 
The pilot’s flip chart listed a final-approach speed of 
130 knots for the DC-9 simulator-configured weight 
of 95 000 lb. Since the announced, simulated surface 
headwind was 8 knots, the resultant expected final- 
approach speed was 134 knots when using the above 
definition. 

Figure 22 shows the mean and the spread of 
indicated airspeed along the entire final-approach 
course for all runs before briefing. The standard de- 
viations of the cockpit-simulator indicated airspeed 
at the half-mile points are also shown. Figure 23 
shows the same information for the runs after brief- 
ing. The airspeed magnitude and spread along the 
final-approach course are similar for both the before- 
and after-briefing runs. The average stabilized speed 
on final approach is approximately 139 knots, which 
is faster than the expected final-approach speed of 
134 knots. The higher average speed would cause 
the individual aircraft to be slightly early relative to 
their scheduled landing time, but it would not affect 
the aircraft-pair interarrival-time error. The spread 
or standard deviation of final- approach speed from 
that expected does affect the aircraft-pair, runway 
interarrival- time- error standard deviation. The data 
of figures 22 and 23 show that the airspeed stan- 
dard deviations were under 5 knots once the final- 
approach speed was established on the glide slope. 
The final-approach data, together with the follow- 
ing analysis at the ILS Category I window, provide a 
basis for modeling individual pilot variation in final- 
approach speed from that expected when performing 
ATC analysis of simulation modeling. 

Figures 24 and 25 show the frequency distribution 
of the final-approach indicated airspeed sampled at 


the ILS Category I window (about 200 ft above and 
2800 ft horizontally from the threshold for a 3° glide 
slope) before and after briefing, respectively. The t 
and F tests even at a 20-percent level of significance 
indicated that both the means and the standard de- 
viations of the final- approach speeds before and after 
briefing were not distinguishable. Consequently, the 
before and after final-approach speeds were combined 
to get the frequency distribution shown in figure 26. 
These pooled data indicated that the final-approach 
speed flown in the simulator was an average 4.8 knots 
faster than expected and the standard deviation was 
3.7 knots. The 68 data approaches flown by the 8 
pilots from 2 airlines support the premise that the 
data of figure 26 are representative of what can be 
expected from airline traffic under conditions similar 
to those simulated. 

4-1-4 Approach Routes and Speeds Flown 

Figure 27 shows the spatial distribution of the 
arrival- approach paths to the final- approach area 
flown by the crews in the data runs along with 
the nominal-approach paths. The aircraft positions 
are shown plotted every simulated radar scan of 
4 sec. As discussed in section 3.1.1, the downwind 
and base legs are flown in response to heading and 
speed instructions from the ATC controller who, in 
turn, was interacting with the TIMER controller aids 
displayed on the PPI. 

Figure 28 shows the base segment of all the 
KEANN data approaches (combined before- and 
after-briefing runs) with the darker position mark- 
ers indicating aircraft locations every 10 scans or 
40 sec. Figure 29 shows the corresponding aircraft 
indicated airspeeds and sample means with the sam- 
ple standard deviations plotted for the corresponding 
40-sec position points. The approach-path spreads 
and speed profiles between the before- and after- 
briefing runs did not differ significantly, and thus 
they were combined in a composite for the KEANN 
approach. In a similar manner, the before- and after- 
briefing runs were combined to show the approach- 
path spread and speed profiles of the DRAKO data 
approaches. These are shown in figures 30 and 31. 
The path divergence and airspeed data are presented 
so that ATC researchers may realistically model pilot 
performance in the terminal area. 

Even though the wind model was not changed 
and each crew was given the same speed instruction 
for the same route, there was some variation in the 
speeds for both approaches flown. The airspeed 
standard deviation a varied between 4 and 7 knots 
along the approaches. Data presented in figures 27 
through 31 in the arrival- approach region and in 
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figures 22 and 23 along the final- approach region give 
a composite model of the aircraft pattern speeds as 
flown in the simulated environment. 

4.1.5 Turn-Command Response Time of 
Aircrew 


4 . 1.5.1 Tum-to- final- command response time of 
aircrew. The timing of the turn-to-final maneuver 
from the base leg is the most crucial approach turn 
with respect to the aircraft arriving at the runway 
threshold at the scheduled landing time. Figures 32 
and 33 show the histograms of pilot responses to ATC 
turn-to-final instructions before and after briefing, 
respectively. The pilot response time is defined as the 
elapsed time between hearing the ATC controller’s 
turn instruction and the time when the aircraft is 
banked into the final turn 5° from the roll attitude 
at the time when the turn instruction was received 
in the cockpit. 

The F test for differences of variance assumes 
Gaussian distributions. The data of figures 32 and 33 
are somewhat skewed. Therefore, a data transforma- 
tion of 

t' = 0.734(4 - 0.9) 0 ' 55 + 4.137 (3) 

was used (ref. 13) on each data point to approximate 
a Gaussian distribution. The F test on the trans- 
formed response-time data before and after briefing 
yielded an F statistic of 


f= m! 

(0.5) 2 


1.44 


( 4 ) 


Even at the 20-percent level of significance, the 
transformed, turn-to-final response-time standard 
deviation after briefing could not be considered differ- 
ent from the standard deviation before briefing. The 
pilots’ mean response times before and after brief- 
ing were equal for both the measured data and the 
transformed data. Therefore, the means of the pilots’ 
response times as well as the standard deviations be- 
fore and after briefing should be considered equal. 

4- 1.5. 2 Turn-to-final- command response-time 
model of aircrew. Since there was no significant sta- 
tistical difference at the base-to-final turn between 
the pilot response times before and after briefing, 
they were combined as shown in figure 34. A three- 
parameter Weibull distribution with probability den- 
sity function 




(t- 7)^ 
V 


( t>r . ; > 0) I 


/(«)= 0 


(*<7) J 
( 5 ) 


was fitted to the combined response-time data for 
the base-to-final turn. A threshold of 0.9 was se- 
lected and the mean and variance of the Weibull dis- 
tribution were set equal to the measured combined 
data in order to solve for the parameters rj and (3 . 
This yielded values of 7 = 0.9, / 3 = 1.1, and 77 = 
2.28. The resultant fitted Weibull density function is 
superimposed on the measured data in figure 34. Ref- 
erence 14 provides justification for using a Weibull 
distribution to model the human response time. 
Thus, we have an analytical expression for the prob- 
ability density function of the pilot’s response time 
to the ATC controller’s turn-to-final instruction. 
The cumulative distribution of the three-parameter 
Weibull distribution is equal to 


Fit) = 1 — exp 

(t-7^1 

{t > 7 ; > o) 

11 

0 

0 

» 

(< < 7) , 


The histogram and fitted-density model of fig- 
ure 34 are the delay responses of 8 airline pilots per- 
forming 62 turn-to-final maneuvers. The model rep- 
resents a simple input /output relationship and does 
not address the detailed contributing factors. There 
are several possible explanations such as variations 
in individual pilot routine response to ATC turn in- 
struction, pilot workload or cockpit activity at the 
time that the turn instructions were issued, and dif- 
ferences in piloting procedures when initiating a turn. 
For example, initiating the turn before setting the 
“bug” on the directional gyro display would produce 
a different response time from the procedure of set- 
ting the bug and then initiating the turn. Another 
controlled and more focused experiment would be re- 
quired to identify and isolate contributing factors. 
Such an experiment could determine if training would 
change the crew’s response-time curve. 

4 . 1.5.3 Approach-turn- command response time 
of aircrew. Figures 35 and 36 show the pilot response 
times to controller turn instructions both before and 
after briefing, respectively, at the first turn on the 
KEANN approach route (©in fig. 11(b)), Corre- 
spondingly, figures 37 and 38 show the response-time 
data before and after briefing for the first turn on 
the DRAKO approach (©in fig. 11(a)). Similarly, 
figures 39 and 40 show the response-time data before 
and after briefing for the second or downwind-to-base 
turn on the DRAKO approach (®in fig. 11(a)). 

For each of these earlier turns, rigorous statistical 
tests on these data are not very enlightening because 
of the limited sample sizes. No statistical differences 
in the pilot response times before and after briefing 
at each of these earlier turns can be claimed. For 
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the same reason, no response-time difference among 
the turns themselves can be rigorously substantiated. 
However, there are a couple of observations about 
the trends in the plotted data that are worth not- 
ing. One is that the response-time scatter for these 
earlier turns in the approach before briefing seems to 
be somewhat reduced after briefing. The other ob- 
servation is that the closer the aircraft gets to the 
runway the more the response-time data tend to get 
skewed with a larger percentage of the reactions con- 
centrated at smaller response times. 

4.1.6 Bank- Angle Performance 

For each of the controller-issued turns during the 
approach, the value of the maximum bank angle 
was recorded because bank angle has an effect on 
time error at the runway. A variation in the bank 
angle will vary the radius of the turn and, thus, the 
distance traveled and the time of arrival. A time 
error will result if the pilot uses a bank angle that 
is significantly different from that assumed in the 
TIMER algorithm. The bank angles flown by the 
test crews during the turn-to-final maneuver from the 
base leg are examined first and in greater detail than 
the other turns since that turn is the most crucial 
relative to runway-arrival time accuracy. Figures 41 
and 42 show the histograms of maximum bank angles 
used in the turn-to-final maneuver before and after 
briefing, respectively. 

There are two noteworthy differences between the 
before- and after-briefing bank-angle distributions for 
the turn-to-final maneuver. The bank-angle den- 
sity is considerably skewed before briefing but is 
more symmetrical after briefing. The other differ- 
ence observed is that the bank angles after briefing 
are more closely bunched. The standard deviation, 
which measures dispersion, is 2.8° after briefing as 
compared with 4.6° before briefing. The F test at 
the 0. 7-percent significance strongly supports the hy- 
pothesis that the subject pilots kept their bank angles 
in a narrower range of values after the briefing while 
performing the base-to-final turn. 

A Weibull distribution, with the probability den- 
sity function defined by equation (5), was fitted to 
the data of figures 41 and 42. The parameters of 
the Weibull distribution before briefing are 7 = 19, 
/3 = 1.51, and 77 = 7.54. The parameters of the 
Weibull distribution after briefing are 7 = 20, /3 = 
2.56, and r) = 7.55. The fitted curves for before and 
after briefing are shown in figures 41 and 42, respec- 
tively. These distributions can be used to model in- 
dividual pilot variation in the selection of bank angle 
for the base-to-final turn. 

Figures 43 and 44 show the maximum bank an- 
gle employed when turning to the base leg on the 


KEANN approach (®in fig. 11(b)) both before and 
after briefing, respectively. The bank-angle standard 
deviations computed were 5.0° before briefing and 
3.2° after briefing. For the values computed be- 
fore and after briefing, the significance must be at 
least 8 percent in order to reject the null hypoth- 
esis (i.e., < t b ef or e = ^after)- Because of the limited 
sample size, there is a less-than-conclusive statisti- 
cal case for the contention that, there was a differ- 
ence in the bank-angle standard deviations before 
and after briefing for the turn-to-base maneuver on 
the KEANN approach. 

Figurs 45 and 46 show the maximum-bank-angle 
data before and after briefing, respectively, for the 
turn to the downwind on the DRAKO approach (® 
in fig. 11(a)). The standard deviation was only 
slightly reduced from 3.8° to 3.2°. There was no 
statistical significance to the bank-angle standard 
deviations before and after briefing on the turn-to- 
downwind maneuver on the DRAKO approach. 

Figures 47 and 48 show the maximum-bank-angle 
data before and after briefing for the turn-to-base leg 
on the DRAKO approach in fig. 11(a)). The 
bank-angle standard deviations computed were 3.4° 
before briefing and 2.2° after briefing. For the values 
computed before and after briefing, the significance 
must be at least 9 percent in order to reject the null 
hypothesis of equal variances. Because of the limited 
sample size there is a less-than-conclusive statistical 
case for the contention that there was a difference 
in the bank-angle standard deviations before and 
after briefing for the turn-to-base leg on the DRAKO 
approach. 

The rigorous statistical test on the earlier termi- 
nal approach turns before the base-to-final turn was 
not as conclusive as would have been desired because 
of the limited sample sizes. However, the trends 
in the plotted data shown in figures 43 through 48 
for these three earlier turns all seem to indicate the 
scatter of the maximum bank angle before briefing 
was reduced somewhat after briefing. There were no 
significant differences between the before- and after- 
briefing performances observed for the other aircraft 
parameters measured such as speed and time re- 
sponses. However, the maximum bank angle was an 
exception. The tendency to reduce the dispersion or 
spread of the bank angle after the pilots received the 
TIMER briefing was clearly evident in the data on 
the base-to-final turn. The reduction in the standard 
deviation of the bank angles measured after briefing 
for the base-to-final turn was apparently enough to 
translate to a slight, though statistically significant 
(at the 5.6-percent level), impact on the delivery- time 
performance at the runway threshold. 
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4-1-7 Final- Approach Data 

To characterize and gather data for modeling of 
crew and airborne system performance during the 
final stages of the flight, various data such as alti- 
tude, vertical tracking of the glide slope beam, lat- 
eral tracking of the localizer beam, and airspeed were 
monitored and recorded continuously. In addition, 
snapshot data were recorded at the ILS Category I 
window and the runway-threshold window. Little or 
no difference was seen between crews, between air- 
lines, or between before and after briefing; therefore, 
only mean and standard deviation plots of all runs 
before briefing are presented for the final-approach 
parameters. For the snapshot data, all data for all 
runs are presented on single plots. 

Figure 49 presents the mean and standard 
deviation plots for all runs/all crews for flights con- 
ducted before briefing. Altitude performance, lo- 
calizer tracking, airspeed performance, and rate-of- 
descent performance are plotted versus distance from 
the runway threshold. The plots begin just before 
intercept of the glide slope beam. The data for the 
flights after briefing are very similar to those pre- 
sented herein. 

Figure 50 presents snapshot data for the glide 
slope error and localizer error for the Category I win- 
dow. The ILS Category I window was 200 ft above 
the ground where the crew must acquire the runway 
environment in order to land. The data presented are 
for all crews and all runs both before and after brief- 
ing. All runs were within acceptable parameters and 
were completed to touchdown. Figure 51 presents 
the same data at the runway-threshold window. 

4-1-8 Crew Questionnaire and Results 

In addition to recording the physical data de- 
scribed above, a rating sheet was administered to the 
captain and first officer at the end of each flight. The 
major objectives of the rating sheets were (1) to de- 
termine if, after briefing, crew concern about prompt 
and consistent response to ATC request raised their 
perceived workload, and (2) to establish whether the 
simulation was realistic and representative of real- 
world conditions. 

The rating sheet contained eleven 7-point scales 
featuring bipolar adjective pairs that dichotomized 
the following descriptors: (1) physical workload 

(low/high), (2) cognitive workload (low/high), 
(3) perceptual workload (low/high), (4) overall work- 
load (low/high), (5) safety (safe/not safe), (6) pas- 
senger acceptance (acceptable/not acceptable), 

(7) skill required (minimum/maximum piloting skill), 

(8) controllability (easy to control/hard to control), 

(9) uneasiness (not uneasy /uneasy), (10) crew mem- 
bers’ performance (satisfactory/unsatisfactory), and 


(11) ATC assessment (identical/very different). In 
addition, space was provided for any additional com- 
ments that the crew members desired to make about 
the flight. Figure 52 presents an example of the rat- 
ing sheet. A list of rating sheet definitions (fig. 53) 
was presented to each crew member before the flight 
began, and this definitions list was available to the 
crew as they filled out the rating sheet after each 
flight. This rating sheet has been developed over 
several studies and was used in its present form with 
a high degree of success in the study discussed in 
references 15 and 16. 

The ratings selected by each crew member from 
the 7-point scale were converted to a number in the 
range of 1 to 7 for each subjective descriptor, where 
a “1” represented the most favorable rating (the low- 
est workload, etc.) and a “7” represented the least 
favorable rating (the highest workload, etc.). Aver- 
age ratings across all captains and the corresponding 
standard deviations were then computed for each of 
the subjective descriptors on the rating sheet. This 
was done both for the flights conducted before brief- 
ing and for the flights conducted after briefing. The 
same data reduction process was performed for the 
first officers. 

In order to ensure that differences seen in the data 
between before briefing and after briefing were due 
to the briefing and not due to training effects, the 
data were tested for training effects using a statistical 
t test. The first flight before briefing for all crews 
was compared with the last flight before briefing for 
all crews. No statistically significant difference was 
detected for any of the descriptors between the first 
flight and the last flight, thus indicating that there 
were no training effects present in the data. 

For the captains, each workload category (fig. 54) 
was rated on the favorable end of the scale for the 
flights before briefing and improved slightly after the 
briefing. A significant difference was detected at 
the 5-percent level between before briefing and after 
briefing for physical workload and perceptual work- 
load. The first officers’ workload ratings for runs 
both before and after briefing indicate less workload 
than the captains’ ratings. Although the first offi- 
cers’ results showed a slight reduction in their work- 
load ratings after briefing, the differences were not 
statistically significant. The important point is that 
the crews’ after-briefing awareness of the importance 
of attention and prompt response to the ATC request 
did not raise the perceived workload and, if anything, 
the challenge may have even lowered their subjective 
perceived workload. 

For the next six categories rated by the captains 
(fig. 54), all categories were rated in the most favor- 
able one-third of the scale with only “skill required” 
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and “crew member performance” rated above 2.3 
before briefing. After briefing, all six categories im- 
proved with only “skill required” rated above 1.9. Al- 
though a significant difference between before brief- 
ing and after briefing was detected at the 5-percent 
level for “controllability” and “uneasiness,” and at 
the 1-percent level for “crew member performance,” 
it should be emphasized that all ratings were in the 
most favorable section of the scale. Bear in mind 
that the ratings are subjective, and thus the general 
range on the scale is more important than the abso- 
lute value. For the first officers (fig. 55), little differ- 
ence was seen in the ratings between before and after 
briefing, and no significant differences were detected. 
Except for “skill required” (which was rated at 2.7), 
the other five categories were rated between 1 and 2. 
There are two significant points to be made from the 
six rating categories discussed above as well as from 
pilot comments. The crews indicated that the ma- 
neuvers were nominal and would cause no passenger 
acceptance problems. In the crews’ judgment, no 
unusual pilot skill would be required to perform the 
TIMER-assisted approaches. 

The final category to be rated was that of “ATC 
assessment,” where an attempt was made to see if the 
crew members perceived any significant differences 
between the simulated ATC environment and the real 
world. As can be seen from the figures, the crew 
members rated the simulated environment virtually 
identical to the real world. 

4.2 ATC Controller’s Message-Delivery 

Performance 

Since the TAATM simulation was originally de- 
signed to provide a realistic environment for cockpit 
research, pseudo-pilot capability to drive all simu- 
lated aircraft was not available at the time of this 
experiment. Lack of pseudo-pilot aircraft control ca- 
pability was acceptable because the primary focus 
of the experiment was to measure the performance 
of conventional aircraft (without a 4D flight man- 
agement system) in a TIMER environment. Con- 
sequently, the pilots in the cockpit simulation re- 
acted to the controller’s verbal command; however, 
the other simulated aircraft were under the control of 
the TIMER real-time program. As discussed in sec- 
tion 3 and shown in figure 7, the verbal commands 
to the other traffic were issued to a pseudo pilot who 
verbally replied and initiated radio contact, but did 
not actually input trajectory commands. 

Figure 56 shows the delivery errors (relative to 
the TIMER expected delivery times discussed in sec- 
tions 2.2 and 3.2.4) of the ATC controller’s final- 
turn instruction to the DC-9 cockpit. The tim- 
ing information obtained depended on the controller 


manually activating an electronic data tablet imme- 
diately after message delivery to halt a computer 
timer that was activated by the DICE countdown. 
There was some human time inaccuracy in activat- 
ing the data tablet relative to message delivery, which 
contributed some imprecision to the controller’s mea- 
sured time errors in message delivery. However, the 
plotted values of figure 56 are a reasonable estimate 
of the time errors by the controller in delivering the 
final-turn instruction to the DC-9 cockpit. The mea- 
sured delivery errors had a mean of 1.0 sec late and 
a standard deviation of 2.1 sec. A three-parameter 
Weibull distribution with a density given by equa- 
tion (5) was fitted to the data as shown in figure 56. 
The parameters of the plotted Weibull distribution 
are 7 = —4.0, (3 = 2.52, and 77 = 5.63. 

Figure 57 shows the controller’s measured time 
errors in delivering the final-turn instruction to the 
TIMER simulated traffic other than the DC-9 cock- 
pit. The measured delivery error had a mean of 
1.8 sec late and a standard deviation of 2.7 sec. A 
three-parameter Weibull distribution with a density 
given by equation (5) was fitted to the data and is 
also shown in figure 57. The parameters of the plot- 
ted curve are 7 = —4.0, (3 = 2.25, and 77 = 6.54. 

The ATC controller was an integral part of the 
TIMER/DC-9 experiment. Therefore, he knew that 
commands issued to other TIMER simulated and 
controlled aircraft, although adding realism to the 
DC-9 cockpit environment, did not affect the trajec- 
tory of that other traffic. Given this situation, the 
subject controller could have inadvertently paid more 
attention to issuing instructions to the DC-9 cockpit 
than would have been the case if all the traffic de- 
pended on his instructions. The data of figures 56 
and 57 support that hypothesis. The delivery- time- 
error standard deviation of the controller's final- turn 
instruction to the DC-9 cockpit (2.1 sec) was com- 
pared with the standard deviation to the TIMER in- 
ternally controlled traffic (2.7 sec). The standard F 
test indicated that at the 2-percent level of signifi- 
cance, there was a difference between the two stan- 
dard deviations. 

It is reasonable to assume that the subject con- 
troller’s final-turn message-delivery performance in 
an operational TIMER environment is bounded by 
the two cases shown in figures 56 and 57. A stan- 
dard deviation of 2.3 sec was used in section 4.1 to 
compute the system interarrival-error performance. 
Although showing the performance of only one con- 
troller, the data represent a credible initial estimate 
of general controller performance. A more rigorous 
experiment to characterize the controller’s perfor- 
mance of message delivery-time-error performance of 
controllers in general in an operational TIMER-like 
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environment would require two additional conditions: 
(1) the trajectory of all simulated aircraft dependent 
upon controller instruction, and (2) the measurement 
of a number of certified and practicing final-approach 
controllers as subjects. 

4.3 Potential TIMER Improvements 

Sections 2 and 3 described the TIMER concept 
and the mode of controller interface used for this 
study. Experience with the reported real-time study 
and discussions with the experimental subjects have 
led to some ideas that have the potential for improv- 
ing performance and acceptance. 

4*3.1 Procedural Changes 

The calculations used by TIMER during the ex- 
periment assume that the pilots follow a procedure 
in which they deploy the landing gear after inter- 
ception of the lower edge of the glide slope, then fully 
deploy the landing flaps. The resultant speed profile 
is a pattern of deceleration across the final-approach 
fix (outer marker) from approach speed to final as 
shown in figures 22 and 23. There is some variation 
in the point where speed reduction begins because of 
altitude and piloting procedure. An alternative pro- 
cedure often used in ATC practice, particularly at 
the busier terminals, is for the controllers to request 
that aircraft maintain speed to the final-approach fix. 
This procedure has the potential of slightly reducing 
the variation of the location where transition from 
approach to final speed begins, thus making the time 
duration along the final path slightly more consistent. 

Headings in the real-time test were given to the 
nearest 10°. A resolution of the heading value to the 
nearest 5° was suggested since that is sometimes used 
in practice. It is true that 5° resolutions would better 
match the aircraft path with the desired ground 
track. However, based on the fast-time sensitivity 
study of reference 1, a reduction of less than 1 sec 
in the interarrival-error standard deviation would be 
expected from flying headings to the nearest 5° . The 
extent of pilot compliance with more precise heading 
instructions is not known; however, subject reaction 
to the proposal indicated no pilot reluctance to flying 
headings with a 5° resolution. 

4*3.2 Controller’s Interface Alternatives 

The current TIMER approach is to transmit the 
suggested commands to the controller via the aircraft 
data tags by adding more fields of information to 
that currently displayed by the ARTS system. The 
authors feel that is the location of choice because it 
is in the normal field of view where the controller’s 
attention can remain focused on the aircraft locations 
and proximity to other aircraft. In addition, when 


action is called for, the particular aircraft involved 
is readily apparent since its data tag contains the 
information. However, there is another point of 
view which holds that the ARTS data tags already 
contain enough information. In addition to possible 
information saturation, the additional fields in the 
data block would magnify tag offset from the aircraft 
symbol in congested conditions. Another approach to 
be explored is to have TIMER suggested commands 
appear in a special position on the PPI screen or in a 
message window. In this format, the messages would 
probably be ordered on their desired delivery times. 

In the real-time experiment the DICE proce- 
dure was implemented by displaying the new de- 
sired heading only after the DICE countdown reached 
zero. The reason is that the calculated desired head- 
ing occasionally changes as a function of wind pro- 
file, aircraft altitude, schedule change, and projected 
runway-centerline intercept distance from the final- 
approach fix. An alternative procedure would be to 
display the projected heading earlier even if some er- 
ror resulted. This trade-off needs further controller 
evaluations to determine if earlier heading display im- 
proves performance or reduces controller workload. 

Another idea that has potential application in 
TIMER command/controller interaction also has 
broader potential applications. Borrowing from the 
military concept of look-to-aim weapons in aircraft, 
an idea was advanced to use an automatic cursor that 
follows controller lookpoint. This technique could 
potentially be quicker and less tiring than a track- 
ball or even a touchscreen. Instruments such as the 
occulometer (ref. 17) could be used as a basis to ex- 
plore this technique. 

5.0 Major Results and Concluding 
Remarks 

Several facilities including a full-workload DC-9 
cockpit and a real-time TIMER (traffic intelligence 
for the management of efficient runway) simulation 
were coupled together. These facilities together with 
a certified air traffic control (ATC) controller, pseudo 
pilot, and airline crew formed the basis of a total 
system simulation for real-time crew-in-the-loop ex- 
periments. Performance data were gathered as the 
crew flew simulated instrument flight rules (IFR) ap- 
proaches with a Denver Stapleton runway 26L con- 
figuration. The following is a summary of the major 
findings. 

Based on the measured real-time cockpit delivery 
error at the runway threshold, a system aircraft-pair 
interarrival-error standard deviation was determined 
to be in the range of 10.4 to 14.1 sec. The 14.1 sec 
results from measurements taken when DC-9 certified 
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airline pilots were asked to fly manual approaches as 
normally done in their company DC-9 aircraft. The 

10.4 sec results from measurements taken after the 
pilots were briefed on the TIMER concept and how 
its performance depended on timely and consistent 
execution of ATC request. The aircraft-pair, runway 
interarrival-error standard deviation ranging from 

10.4 to 14.1 sec supported the 12 sec predicted by 
earlier TIMER fast- time simulations. 

There was a slight improvement in the delivery 
performance after the crew briefing. Presumably, the 
after-briefing situation represented pilots with knowl- 
edge and awareness of the importance of airborne 
crew performance, i.e., pilots that were motivated 
and attentive. What is significant about these re- 
sults is how well the TIMER system worked even 
with normal pilot response to ATC instructions. As 
a point of reference, a field study by Martin and 
Willett in 1968 measured 26.5 sec as the aircraft- 
pair, runway interarrival-error standard deviation of 
manual control with no computer aiding. The real- 
time, human-in-the-loop simulation results of this 
study, as well as earlier fast-time results, show the 
potential of a time-based, terminal-area flow control 
system with controller aids such as those obtained 
in TIMER. With ground knowledge of the aircraft 
final-approach speed, the non-four-dimensional (non- 
4D), runway interarrival-error standard deviation of 
an aircraft pair could be reduced to a region between 

10.4 and 14.1 sec. 

The extent of pilot-induced variation in final- 
approach speed from that expected for an aircraft 
type and landing weight is an important parameter in 
assessing a terminal time- based flow control system 
such as TIMER. The means and standard deviations 
of the final- approach speeds before and after briefing 
were not distinguishable. The pilot-induced variation 
in the final- approach speed flown in the simulator 
was an average 4.8 knots faster than expected, and 
the standard deviation was 3.7 knots. These data 
are significant because of the current limited data 
base for the modeling of pilot /aircraft performance 
in the field of ATC system analysis and simulation 
modeling. 

The measured pilot response times to the turn- 
to-final instruction before and after briefing were 
not distinguishable. The before- and after-briefing 
data were pooled and fitted with a three-parameter 
Weibull distribution. Thus, an analytical expres- 
sion was obtained for the probability density and 
the cumulative distribution of pilot response time 
to the ATC controller’s turn-to-final instruction. 
This model should be used in ATC system analy- 
sis and simulation modeling rather than a Gaussian 


distribution because of the considerable skewness of 
the pilot’s response times. 

There are two noteworthy differences between the 
before- and after-briefing bank angles used by the 
pilots during their turn-to-final maneuver. The bank- 
angle density is considerably skewed before briefing, 
but the after-briefing density is almost symmetrical. 
The other difference is that of scatter — the bank- 
angle standard deviation was 4.6° before briefing 
and reduced to 2.8° after briefing. The reduced 
scatter of the bank-angle values after briefing was 
apparently the reason for the slight improvement in 
delivery precision measured after crew briefing. A 
distribution was fitted to the bank-angle data that 
can be used for analysis and computer simulation. 

There are a few points worth mentioning about 
results from the questionnaire administered to both 
the captain and first officer after each simulated 
flight. The crews felt strongly that the ATC instruc- 
tions and simulation scenarios, both before and af- 
ter briefing, were representative and close to realis- 
tic real-world conditions. The crews’ after-briefing 
awareness of the importance (relative to precise 
runway-threshold delivery time) of attention and 
prompt response to an ATC request did not raise 
their perceived workload, and, if anything, the chal- 
lenge may have even lowered the crews’ subjective 
perceived workload. The crews indicated that the 
maneuvers were nominal and would cause no passen- 
ger acceptance problems. In the crews’ judgment, no 
unusual pilot skill would be required to perform the 
TIMER-assisted approaches. 

Because of the nature of the simulation, only the 
DC-9 cockpit reacted to the controller’s verbal com- 
mands. The other traffic was under internal TIMER 
control. Thus, the controller’s time errors (relative 
to the TIMER expected time) in delivering the final- 
turn instruction were separated into errors to the 
DC-9 cockpit and errors to the other TIMER traf- 
fic. A three-parameter Weibull distribution was fit- 
ted to these two data sets. The controller’s message 
delivery-time error to the DC-9 cockpit had a mean of 
1.0 sec late and a standard deviation of 2.1 sec. The 
controller’s time error to the internally controlled 
TIMER aircraft had a mean of 1.8 sec late and a 
standard deviation of 2.7 sec. If all the traffic had 
been controlled by responding live crews, it is felt 
that the controller’s performance would lie between 
the two measured cases above. 

NASA Langley Research Center 
Hampton, VA 23665-5225 
February 6, 1990 
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Horizon of 
control 



Figure 1. Sequence of events that an arrival aircraft would experience in TIMER concept. 




Figure 2. Terminal geometry simulated for approaches to runway 26L at Denver’s Stapleton International 
Airport. 
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Pseudo-pilot station ATC controller station ^ ^ 

Figure 3. The Mission-Oriented Terminal- Area Simulation (MOTAS) Facility. 
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Figure 5. The Visual Landing Display Simulator. 
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Figure 6. Simulation air traffic controller stations. 
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AIRCRAFT CONTROL INTERFACE 
SIMULATION DATA EXCHANGE 

Figure 7. Interface diagram for experimental voice communication, aircraft control, and data exchange. 
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Figure 8. Video map and aircraft positions on controller’s display. 
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DRAKO ARRIVAL - RUNWAY 28L 


DENVER ARR CON - 120.2 

DENVER APP CON - 125.3 

DENVER TOWER - - - 1 18.3 
ATIS 125.6 




N 


VAR 8.5° W 



Figure 9. Nominal frequencies, speeds, headings, and altitudes for DRAKO arrival route of the 
Denver/ Wallops terminal simulation area used in real-time simulation runs. 
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KEANN ARRIVAL 


DENVER ARR CON - 120.2 

DENVER APP CON - 125.3 

DENVER TOWER - - - 118.3 
ATIS 125.6 




Figure 10. Nominal frequencies, speeds, headings, 
Denver/ Wallops terminal simulation area used in r 
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RUNWAY 28 L 






DRAKO approach 



-# 

OM 1900' 


(a) DRAKO approach route. 



OM 1900' 


(b) KEANN approach route. 

Figure 11. Event locations for two arrival routes flown in cockpit simulator. 
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Data-tag fields 


Field 1 DICE value 


(e.g., +75 
-5 


75 sec early at way point 
5 sec late at way point) 


Field 2 Aircraft flight identification 
Field 3 Symbol (H) for heavy jet 
Field 4 Altitude / 1 00 

Field 5 Handoff symbol 

Field 6 Ground speed / 10 

Field 7 Vector command (e.g., L350 

Field 8 Speed comand (e.g., S190 

Field 9 Countdown/countup time for field 7 

Field 10 Countdown/countup time for field 8 


Left turn to 350°) 
Speed 190 knots) 


Figure 12. Aircraft data-tag information for TIMER display. 
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UA422 Normal ARTS III Data Block: Top line - Aircraft call sign (United 422); 

Bottom line - altitude in feet / 100 (11000 ft), and ground speed in 
1 10 23 knots / 10 (230 knots). 


UA422 
1 10 23 
LI 90 


Command added on line below normal data block - 24 sec prior to 
transmittal time, the command (in this case a left turn to 190°) 
appears . 


UA422 

110 23 Command begins flashing 12 sec prior to transmittal time. 


UA422 
1 10 23 

LI 90 

- ^ 

8 


Countdown time appears on line below command, 8 sec prior 
to transmittal time; the command continues to flash. 


UA422 
1 10 23 
LI 90 
0 


Countdown reaches zero; the command stops flashing; the command 
should be transmitted now; when transmission is made, a computer 
entry by the controller is required to remove the command and time 
from the data block. 


UA422 
1 10 23 
LI 90 

s'- 


Assuming that the computer entry is not made, the command continues to 
be displayed and the elapsed time since the transmission should have 
been made is indicated; additionally, the time will flash. 


UA422 
1 10 23 


After the computer entry has been made, indicating that the controller 
no longer wants the information to be displayed, a normal data block 
returns. 


Figure 13. Example of a series of data-tag information illustrating a vector control command. 
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AA641 
110 23 


Normal ARTS III data block. 


+039 
AA641 
110 23 

+035 
AA641 
1 10 23 


DICE value appears on line above the normal data block. 


Dice value decrements according to a comparison of the 
scheduled landing time to estimated aircraft arrival time. 
(Arrival time is a function of aircraft performance, tracker- 
estimated aircraft position, and estimated winds.) 


(DICE value continues decrementing, 


+30, +025, +020 . . .) 


+000 
AA641 
110 23 
L350 


DICE value goes to zero; the appropriate command appears 
in the line below the normal data block. 


AA641 
110 23 
L350 
-1 


Elapsed time since the command should have been delivered 
appears in the line below the command; this count continues 
until the controller makes the appropriate computer entry. 


AA641 After the computer entry has been made, indicating that 

* * n 90 the controller no longer wishes that the information be 

' ^ displayed, a normal data block returns. 


Figure 14. Example of a series of data-tag information illustrating a DICE value countdown. 



1 . SET FD MODE/SEL OFF 

2. SET AUTOTHROTTLE ON FOR TRIMMING 

3. SET GEAR UP 

4. SET FLAPS = 0°, SLATS = EXT 

5. SET COMM HEAD #1 TO 120.2 

6. SET NAV HEAD #1 TO 117.0 

7. SET HSI COURSE 

(A) TO 246° (ROUTE #1) 

(B) TO 153° (ROUTE #4) 

8. TELL COMPUTER OPERATOR - READY FOR INITIAL CONDITION S 

9. AFTER TRIM OCCURS AND RUN STARTS, DISENGAGE AUTOTHROTTLE 
AND FLY MANUAL THROTTLES 

10. IF DESIRED SET FD MODE/SEL TO NAV/LOC 

11. AT APPROPRIATE TIME DURING FLIGHT, TUNE ILS, NAV HEAD #1 TO 
1 10.3 AND SET FD MODE/SEL TO APPR AUTO 

12. TUNE COMM HEAD #1 AS DIRECTED BY ATC 

13. CONFIGURE FOR APPROACH AT APPROPRIATE TIMES, USE FLAPS = 40° 
CONFIGURATION FOR LANDING, FLIP-CHART LANDING SPEED = 130 KNOTS 

Figure 15. Simplified cockpit simulator procedure for a run. 
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PROFILE DESCENT RUNWAYS 10-19-28- 1 

Figure 16 . Profile descent navigation chart of terminal area simulated. 
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Figure 17. ILS approach plate of terminal area simulated. 





Delivery error at runway threshold, sec 


Figure 18 . Frequency distribution of single-aircraft 
TIMER briefing to crew. 
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1 0 



Delivery error at runway threshold, sec 


Figure 20. Frequency distribution of single-aircraft, final- approach- fix delivery-time errors for runs before 
TIMER briefing to crew. 



Delivery error at runway threshold, sec 

Figure 21. Frequency distribution of single-aircraft, final-approach-fix delivery-time errors for runs after 
TIMER briefing to crew. 
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Figure 22. Distribution of indicated airspeeds for DC-9 simulator along final approach for runs before 
TIMER briefing to crew. The y distance is relative to TIMER north axis. 
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Indicated airspeed at Category I window, knots 

Figure 24. Frequency distribution of indicated airspeed at ILS Category I window for runs before TIMER 
briefing to crew. 



indicated airspeed at Category I window, knots 

Figure 25. Frequency distribution of indicated airspeed at ILS Category I window for runs after TIMER 
briefing to crew. 
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Indicated airspeed at Category I window, knots 

Figure 26. Frequency distribution of indicated airspeed at ILS Category I window for combined (before and 
after) TIMER briefing runs. 
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Figure 27, Superimposition of arrival paths flown to final-approach area for all data runs. 
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Pilot response time, sec 
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Figure 32. Frequency distribution of turn-to-final response time for runs before TIMER briefing to crew. 
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Pilot response time, sec 

Figure 33. Frequency distribution of turn-to-final response time for runs after TIMER briefing to crew. 
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Pilot response time, sec 

Figure 35. Frequency distribution of KEANN turn-to-base response time for runs before TIMER briefing 
to crew. 
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Figure 36. Frequency distribution of KEANN turn-to-base response time for runs after TIMER briefing 
to crew. 
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Pilot response time, sec 

Figure 37. Frequency distribution of DRAKO turn-to-downwind response time for runs before TIMER 
briefing to crew. 



Pilot response time, sec 

Figure 38. Frequency distribution of DRAKO turn-to-downwind response time for runs after TIMER 
briefing to crew. 
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Figure 39. Frequency distribution of DRAKO turn-to-base response time for runs before TIMER briefing 
to crew. 



Pilot response time, sec 

Figure 40. Frequency distribution of DRAKO turn-to-base response time for runs after TIMER briefing 
to crew. 




Figure 41. Histogram of tum-to-final bank angles for runs before TIMER briefing to crew with fitted 
Weibull density. 



Figure 42. Histogram of turn-to-final bank angles for runs after TIMER briefing to crew with fitted 
Weibull density. 
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Observed frequency 



Figure 43. FYequency distribution of KEANN turn-tcnbase bank angle for runs before TIMER bnefing to 


crew. 



Maximum bank angle during turn, deg 


Figure 44. Frequency distribution of KEANN turn-to-base bank angle for runs after TIMER briefing to 
crew. 
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Maximum bank angle during turn, deg 

Figure 47. Frequency distribution of DRAKO turn-tobase bank angle for runs before TIMER briefing to 


crew. 



Figure 48. Frequency distribution of DRAKO turn-to-base bank angle for runs after TIMER briefing to 




(a) Altitude profile. 
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(b) Localizer deviation. 

Figure 49. Mean and standard deviation of final-approach data for all runs before TIMER briefing to 
crew. 
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(c) Indicated airspeed. 
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(d) Altitude rate of change. 
Figure 49. Concluded. 



FLIGHT TRAJECTORY RATING SHEET 
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Additional Comments: 



RATING SHEET DEFINITIONS 


1. Physical Workload : adjusting, dialing, holding, pressing, pulling, 

pushing, reaching, turning, tuning, talking, 
writing, etc. 

2. Cognitive Workload : thinking, deciding, calculating, estimating, 

judging, checking, planning, timing, predicting, 
etc. 

3 * Perceptual Workload : looking, scanning, searching, listening, feeling, 

noticing, comparing, identifying, matching, etc. 
Receiving and comprehending information through any 
of the senses. 

4. Overall Workload : combination of physical, cognitive, and perceptual 

workload 

5. Safety : Refers to your ability to control the aircraft and 

respond to ATC commands without jeopardizing the 
lives of the passengers and crew. 

Passenger Acceptance : To what extent do you think a typical passenger 

would find the previous flight unacceptable enough 
to express his/her dissatisfaction in some tangible 
way (complain to a member of the crew, complain to 
another passenger, cry out, become ill, protest to 
the airline, comment about the flight procedures to 
friends/relatives, select an alternative means of 
transportation for his/her next trip, etc.). 

2* Skill Required : How much special training or practice do you think 

is necessary to fly approaches/departures such as 
the previous flight, assuming the same avionics and 
flight controls? 

8. Controllability : How easy was it for you to follow the ATC commands 

during the last flight? 

9. Uneasiness : To what extent did you feel that the last flight 

placed you under pressure or subjected you to 
feelings of anxiety, frustration, nervousness, 
stress, etc.? 

10. Your Pe rformance : To what extent do you think your performance during 

the last flight approached the best you are capable 
of doing? Consider whether or not you think your 
performance on the last flight would have been 
significantly better if you had had additional 
practice/training. 

ATC Assessment : How close to normal IFR terminal approach procedures 

do you judge the ATC commands and procedures to be? 

Figure 53. Definition of rating categories on crew evaluation sheet. 
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Figure 54. Subjective ratings of captains. 
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Figure 55. Subjective ratings of first officers. 





Figure 56. Histogram of controller’s final-turn-message delivery-time error to DC-9 simulator and fitted 
Weibull density. 



Figure 57. Histogram of controller’s final-turn-message delivery-time error to TIMER-controlled aircraft 
and fitted Weibull density. 
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